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Abstract 

We investigate the optimal performance of dense sensor networks by studying the 
joint source-channel coding problem. The overall goal of the sensor network is to 
take measurements from an underlying random process, code and transmit those mea- 
surement samples to a collector node in a cooperative multiple access channel with 
potential feedback, and reconstruct the entire random process at the collector node. 
We provide lower and upper bounds for the minimum achievable expected distortion 
when the underlying random process is Gaussian. When the Gaussian random process 
satisfies some general conditions, we evaluate the lower and upper bounds explicitly, 
and show that they are of the same order for a wide range of power constraints. Thus, 
for these random processes, under these power constraints, we express the minimum 
achievable expected distortion as a function of the power constraint. Further, we show 
that the achievability scheme that achieves the lower bound on the distortion is a 
separation-based scheme that is composed of multi-terminal rate-distortion coding and 
amplify-and-forward channel coding. Therefore, we conclude that separation is order- 
optimal for the dense Gaussian sensor network scenario under consideration, when the 
underlying random process satisfies some general conditions. 
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1 Introduction 



With the recent advances in the hardware technology, small cheap nodes with sensing, com- 
puting and communication capabilities have become available. In practical applications, 
it is possible to deploy a large number of these nodes to sense the environment. In this 
paper, we investigate the optimal performance of a dense sensor network by studying the 
joint source-channel coding problem. The sensor network is composed of sensors, where 

is very large, and a single collector node. Each sensor node has the capability of taking 
noiseless samples from the underlying random process, and is equipped with one transmit 
and one receive antenna to transmit and receive signals. The overall goal of the sensor 
network is to take measurements from an underlying random process S{t), < t < Tq, 
code and transmit those measured samples to a collector node, and reconstruct the entire 
random process at the collector node, with as little distortion as possible; see Figure [H Due 
to the small distances between the sensor nodes and the correlation in the measured data, 
the underlying sources are correlated, and due to the existence of receive antennas at the 
sensor nodes and a transmit antenna at the collector node, the communication channel is a 
Gaussian cooperative multiple access channel with potential feedback. We investigate the 
minimum achievable expected distortion and the corresponding achievability scheme when 
the underlying random process is Gaussian. 

Following the seminal paper of Gupta and Kumar [3], which showed that multi-hop 
wireless ad-hoc networks, where users transmit independent data and utilize single-user 
coding, decoding and forwarding techniques, do not scale up, Scaglione and Servetto [4] 
investigated the scalability of the sensor networks. Sensor networks, where the observed 
data is correlated, may scale up for two reasons: first, the correlation among the sampled 
data increases with the increasing number of nodes and hence, the amount of information the 
network needs to carry does not increase as fast as in ad-hoc wireless networks; and second, 
correlated data facilitates cooperation, and may increase the information carrying capacity 
of the network. The goal of the sensor network in [4] was that each sensor reconstructs the 
data measured by all of the sensors using sensor broadcasting. In this paper, we focus on 
the case where the reconstruction is required only at the collector node. Also, in this paper, 
the task is not the reconstruction of the data the sensors measured, but the reconstruction 
of the underlying random process. 

Gastpar and Vetterli [5] studied the case where the sensors observe a noisy version of a 
linear combination of L Gaussian random variables which all have the same variance, code 
and transmit those observations to a collector node, and the collector node reconstructs the 
L random variables. In [5], the expected distortion achieved by applying separation-based 
approaches was shown to be exponentially worse than the lower bound on the minimum 
achievable expected distortion. In this paper, we study the case where the data of interest 
at the collector node is not a finite number of random variables, but a random process. 
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Figure 1: Sensor network. 

which, using Karhunen-Loeve expansion, can be shown to be equivalent to a set of infinitely 
many random variables with varying variances. We assume that the sensors are able to take 
noiseless samples, but that each sensor observes only its own sample. Our upper bound on 
the minimum achievable distortion is also developed by using a separation-based approach, 
but it is shown to be of the same order as the lower bound, for a wide range of power 
constraints, for random processes that satisfy some general conditions. 

El Gamal [6] studied the capacity of dense sensor networks and found that all spatially 
band-limited Gaussian processes can be estimated at the collector node, subject to any 
non-zero constraint on the mean squared distortion. In this paper, we study the minimum 
achievable expected distortion for space-limited, and thus, not band-limited, random pro- 
cesses, and we show that the minimum achievable expected distortion decreases to zero as 
the number of nodes increases, unless the power constraint is unusually small. Also, in [6], 
it is assumed that the channel gains between the nodes decrease with the distance between 
them, without enforcing any upper bounds. This implies that, when the sensors are placed 
very densely, the channel gains between nearby sensors become unboundedly large. This 
physically impossible situation arises because although the channel model used in [6] is vahd 
only in the far field of the transmitter, it is used for all distances. Although we adopted this 
channel model in [1] , we have changed the channel model to a more realistic one in this paper 
(and [2]), where we assume that the channel gains decrease with distance, however, they are 
lower and upper bounded. The difference in the channel models in [1] and here (and [2]), 
does not affect our conclusion, i.e., in both cases, we are able to find achievable schemes 
that achieve the lower bound on the distortion. However, it affects the achievability scheme 
itself; in [1] the achievability scheme is based on the basic idea of decode-and-forward as the 
channel model allows a significant number of nodes to be able to decode successfully the 
signal transmitted by a node, whereas the achievability scheme here (and [2]) is based on the 
basic idea of amplify-and-forward, where due to the lower and upper bounds on the channel 
gains a sufficient amount of beamforming effect is achieved through the amplify-and-forward 
scheme. 
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From an information theoretic point of view, our problem is a joint source-channel coding 
problem for lossy communication of correlated sources over a cooperative Gaussian multiple 
access channel with potential feedback. The simpler problem of lossless reconstruction of cor- 
related sources over a multiple access channel without cooperation or feedback still remains 
open [7-10]. Therefore, a direct and closed-form expression for the distortion seems unlikely 
to be obtained, and consequently, we resort to developing lower and upper bounds. We first 
provide lower and upper bounds for the minimum achievable expected distortion for arbi- 
trary Gaussian random processes whose Karhunen-Loeve expansion exists. Then, we focus 
on the case where the Gaussian random process also satisfies some general conditions. For 
these random processes, we evaluate the lower and upper bounds explicitly, and show that 
they are of the same order, for a wide range of power constraints. Thus, for these random 
processes, under a wide range of power constraints, we determine the order-optimal achiev- 
ability scheme, and identify the minimum achievable expected distortion. Our achievability 
scheme is separation-based: each sensor first performs multi-terminal source coding [11], 
then, performs channel coding, and utilizes the cooperative nature of the wireless medium 
through the amplify-and- forward scheme [12]. In multi-user information theory, generally 
speaking, the separation principle does not hold. However, in our case, we have found a 
scheme which is separation based, and is order-optimal. 



2 System Model 

The collector node wishes to reconstruct a random process S(t), for < t < Tq, where t 
denotes the spatial position; S(t) is assumed to be Gaussian with zero-mean and a continuous 
autocorrelation function K{t, s). The sensor nodes are placed at positions = ti < ^2 < 
■ ■ ■ < t/vT = To, and observe samples Sn = {S{ti), S{t2), ■ ■ ■ , S'(tAr)). For simplicity and to 
avoid irregular cases, we assume that the sensors are equally spaced, i.e., 

^^ = ^^^0, t = l,2,---,N (1) 
The distortion measure is the squared error, 

d{s{t),s{t)) = ^ r\s{t) - m?dt (2) 
-i-o Jo 

Each sensor node and the collector node, denoted as node 0, is equipped with one transmit 
and one receive antenna. To simplify the presentation, from now until Section U\ we will 
assume that the collector node does not use its transmit antenna, and thus, there is no 
feedback in the system. We will allow the collector node to use its transmit antenna and 
provide feedback to the sensor nodes in Section [71 and show that the results of the previous 
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sections remain unchanged. At any time instant, let Xj denote the signal transmitted by 
node i, and Yj denote the signal received at node j. Let hij denote the channel gain from 
node i to node j. Then, the received signal at node j can be written as, 

N 

Y,= J2 h^,X, + Z„ j = 0,l,2,---,iV (3) 

where {Zj}jL(^ is a vector of + 1 independent and identically distributed, zero-mean, unit- 
variance Gaussian random variables. Therefore, the channel model of the network is such 
that all nodes hear a linear combination of the signals transmitted by all other nodes at that 
time instant. We assume that the channel gain is bounded, i.e., 

hi<hi,<hu, i = l,--- ,N, J = 0, 1, ■ ■ ■ , iV (4) 

where hu and hi are positive constants independent of N. This model is very general and 
should be satisfied very easily. By the conservation of energy, hfj < 1, and since all nodes 
are within finite distances of each other, the channel gains should be lower bounded as well. 

We assume that all sensors have the same individual power constraint P{N)/N, where 
P{N), which we will call the total power, is the sum of the individual power constraints, and 
it is a function of N. The two most interesting cases for P{N) are P{N) = XPind where each 
sensor has its individual power constraint Pind, and P{N) = Ptot where the total power is a 
constant Ptot and does not depend on the number of sensors. In the latter case, when more 
and more sensor nodes are deployed, the individual power of each sensor node decreases as 
Ptot/N. Our goal is to determine the scheme that achieves the minimum achievable expected 
distortion at the collector node for a given total power P{N), and also to determine the 
rate at which this distortion goes to zero as a function of the number of sensor nodes and 
the total power. 

Next, we give a more precise definition of our problem. Each sensor node observes a 
sample of a sequence of spatial random processes {S^^\t)}f^^ i.i.d. in time, where index / 
denotes time, t denotes the spatial position, and n is the block length of the sequence of 
random processes, and also the delay parameter. For now, we assume that n channel uses 
are allowed for n realizations of the random process; the case where we allow the number of 
channel uses and the number of observations to differ will be treated in Section [71 At time 
instant m, sensor node j transmits 

X,{m) = Fj-\{S^^\t,)}t^AYj'^}T=i'), m = l,2,---,n, j = l,2,---,iV (5) 

i.e., it transmits a signal that is a function of its observations of the entire block of random 
process samples and also the signal it received before time m. We are interested in the per- 
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formance in the information-theoretic sense and hence, we allow the delay n to be arbitrarily 
large. By the assumption of identical individual power constraints, we have 



if;^[X|(m)]<:^, j = l,2,---,iV (6) 
n ^ — ' iV 

m=l 

The collector node reconstructs the random process as 

{S^^\t),te [0,To]}r=i = GiY^'\Y^'\ . . . ,Yt^) (7) 

For fixed encoding functions of the nodes {Fj^^}'^ZijZi and the decoding function of the 
collector node G, the achieved expected distortion is 

l-J2E[d{si^\t),S^^\t))] (8) 

1=1 

and we are interested in the smallest achievable expected distortion over all encoding and 
decoding functions where n is allowed to be arbitrarily large. 

In this paper, our purpose is to understand the behavior of the minimum achievable 
expected distortion when the number of sensor nodes is very large. We introduce the big-0 
and big-© notations. We say that / is 0{g), if there exist constants c and k, such that 
\f{^)\ < c\g{N)\ for all N > k; we say that / is Q{g), if there exist constants Ci, C2 and 
k such that ci\g{N)\ < \f{N)\ < C2|^(A^)| for all N > k. All logarithms are defined with 
respect to base e, and [x\ denotes the largest integer smaller than or equal to x. 

3 A Class of Gaussian Random Processes 

For a Gaussian random process S{t) with a continuous autocorrelation function, we perform 
the Karhunen-Loeve expansion [13], 

oo 

S{t) = J2SkMt) (9) 

fe=0 

to obtain the ordered eigenvalues {Xk}kLo, and the corresponding eigenfunctions {(f)k{t),t e 

[o,To]}r=o- 

Let A be the set of Gaussian random processes on [0, To] with continuous autocorrelation 
functions, that satisfy the following conditions: 

1. There exist nonnegative constant d and nonnegative integers ci, Cu, Kq > Cu + 1 and 
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two sequences of numbers {A'^}^q and {A^'}^q defined as 



K = { '\ f^-j, (10) 



and 



K = { 'I^ -2 (11) 



for some constant x > 1, such that 



a; < a, < (12) 



The condition that x > 1 is without loss of generahty, because for all continuous 
autocorrelations, the eigenvalues decrease faster than k~^. 

2. In addition to continuity. K{t, s) satisfies the Lipschitz condition of order 1/2 < a < 1, 
i.e., there exists a constant B > such that 

\K{t,, s,) - K{h, S2)\<B ( - + (si - s^^y (13) 

for all ti, si, ^2, -52 e [0, To]. 

3. For k = 0, 1, • ■ ■ , the function c6fc(s) and the function K{t, ,s)(;6j.(s) as a function of s 
satisfy the following condition: there exist positive constants Bi, B2, B3, B4, /3 < 1, 
7 < 1, and nonnegative constant r, independent of k, such that 

\Msi) - Ms2)\ < B^{k + B^Ylsi - (14) 

and 



\K{t, Si)Msi) - K{t, S2)MS2)\ < B2{k + BiYlsi - S2f (15) 

for all t,Si,S2 e [0,ro]. 

The reasons why these conditions are needed for the explicit evaluation of the lower and 
upper bounds on the minimum achievable expected distortion will be clear from the proofs. 
Here, we provide some intuition as to why they are needed. Condition 1 states that we 
consider random processes that have eigenvalues A^ which decrease at a rate of approximately 
k~^. The rate of decrease in the eigenvalues is an indication of the amount of randomness 
the random process contains. Thus, the minimum achievable expected distortion depends 
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crucially on the rate of decrease parameter x. The lower (upper) bound on the eigenvalues in 
(fT2|) will be used to calculate the lower (upper) bound on the minimum achievable expected 
distortion. Conditions 2 and 3 are needed because instead of the random process itself that 
is of interest to the collector node, the collector node, at best, can know only the sampled 
values of the random process. How well the the entire process can be approximated from 
its samples is of great importance in obtaining quantitative results. Lipschitz conditions 
describe the quality of this approximation well. By condition 3, we require the variation in 
the eigenfunction 0^ to be no faster than k'^. We note that the well-known trigonometric 
basis satisfies this condition. 

We also note that our conditions are quite general. Many random processes satisfy 
these conditions, including the Gauss-Markov process, Brownian motion process, centered 
Brownian bridge, etc. For example, a Gauss-Markov process, also known as the Ornstein- 
Uhlenbeck process [14, 15], is defined as a random process that is stationary, Gaussian, 
Markovian, and continuous in probability. It is known that the autocorrelation function of 
this process is [16-18] 

K{t, s) = ^e-''l*-^l (16) 
The Karhunen-Loeve expansion of the Gauss-Markov process yields the eigenfunctions 



kit) = hk I cos - r,H + ^'"^ V £ ~ "^^^ ' ^^^^ 

where {A^j^Q are the corresponding eigenvalues and are positive constants chosen such 
that the eigenfunctions (pkit) have unit energy. It can be shown that {A^j^Q may be bounded 

as 

K < A. < K (18) 



where {A'^}^^ is defined as 



, \u, k < Kn 
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with Kq = max ( 2 



V -'o 
■^2 



and {X'l}'^^^ is defined as 



K 



(k-iy 



k<Ko 
k> Kq 



(20) 



Thus, we observe that the Gauss-Markov process satisfies the conditions defined in this 
section with x = 2 and a = /3 = r = 7 = l. In fact, in a preliminary conference version 
of our work [1], we focused specifically on the Gauss-Markov process and presented results 
similar to those here. We also note, as discussed in the Introduction section, that the channel 
model in [1] is somewhat different than here, and therefore the order-optimal achievability 
schemes in [1] and here are different. 

The lower and upper bounds on the minimum achievable expected distortion will be 
calculated using {A';,}^q and {A';.'}^q, respectively. Some properties of {A'^}^q and {A^'}^q 
which will be used in later proofs are stated in Lemmas [5] and [6] and proved in Appendix 19. II 



4 A Lower Bound on the Achievable Distortion 
4.1 Arbitrary Gaussian Random Processes 

A lower bound is obtained by assuming that all of the sensor nodes know the random 
process exactly, and, the sensor network forms an iV-transmit 1-receive antenna point-to- 
point system to transmit the random process to the collector node. Let be the capacity of 
this point-to-point system and Dp{R) be the distortion-rate function of the random process 
S(t) [19]. In this point-to-point system, the separation principle holds, and therefore 

D"" > D,{C^) (21) 

To evaluate Dp{C^), we first find the distortion-rate function, Dp{R), of S(t) [19, Section 
4.5] as, 

i?(0) = f;max(^O,^log(^)) (22) 

and 

oo 

Die)=T,-'Y,^^i^,^k) (23) 

k=0 
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Next, we find C^, the capacity of tlie iV-transmit 1-receive antenna point-to-point system [20] 
as, 



(24) 



To see liow clianges witli N , using flM|) and (jl]), we can lower and upper bound as 

\ log (1 + h'iNP{N)) < < i log (1 + hlNP{N)) (25) 

For arbitrary Gaussian random processes, a lower bound on the minimum achievable ex- 
pected distortion is 

= D,{C^) (26) 

4.2 The Class of Gaussian Random Processes in A 

Next, we evaluate Dp{C^) for the class of Gaussian random processes in A. Based on the 
structure of the eigenvalues in ( |T0|) and ( |T2|l . and the properties of {A'^}^q in Lemma [5] in 
Appendix 19.11 we have the following lemma. 

Lemma 1 For Gaussian random processes in A, for any constant Q < n < 1, we have 

m > ^O-i (27) 

om > . (i + ^) I"'-* P«) 

when 9 is small enough . 

A proof of Lemma [T] is provided in Appendix 19. 2[ 

Using Lemma [H we present in the next theorem a lower bound for the distortion-rate 
function of the random process. 

Theorem 1 For Gaussian random processes in A, for any constant Q < n < 1, we have 
when R is large enough. 

A proof of Theorem [1] is provided in Appendix 19.31 
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We will divide our discussion into two separate cases based on the total power, P{N). 
For the first case, P{N) is such that 

lim T77T7T7T = (30) 

N-.00 NP{N) ^ ' 

The cases P{N) = NP-.^d and P{N) = Ptot are included in P{N) satisfying fl30|). 

Theorem 2 For Gaussian random processes in A, when P{N) is such that ^3^) is satisfied, 
for any constant < k, < 1, a lower bound on the minimum achievable expected distortion is 

= DJC^) >t^(l + [t^xf-^ - ( — y ' (31) 

when N is large enough. 

A proof of Theorem [2] is provided in Appendix 19.41 

Hence, when total power P{N) satisfies (130|) . a lower bound on the achievable distortion 

is 

«((Mivk^V"l (32) 

For the second case, P{N) is such that ([30]) is not satisfied. In this case, is either a 
constant independent of or goes to zero as A^ goes to infinity. The minimum achievable 
distortion does not go to zero with increasing A^. 

Therefore, for all possible total power P{N), a lower bound on the distortion is 

2 \ x—i 



« I 1 1 Sii™ j -Vj '''' 

When the total power P{N) grows almost exponentially with the number of nodes, 
the lower bound on the minimum achievable expected distortion in fl33p decreases inverse 
polynomially with A^. Even though this provides excellent distortion performance, it is 
impractical since sensor nodes are low energy devices and it is often difficult, if not impossible, 
to replenish their batteries. When the total power P{N) is such that fl30|) is not satisfied, 
the transmission power is so low that the communication channels between the sensors and 
the collector node are as if they do not exist. From ( 133|) . the lower bound on the estimation 
error in this case is on the order of 1, which is equivalent to the collector node blindly 
estimating S(t) = for all t G [0,To]. Even though the consumed total power P{N) is 
very low in this case, the performance of the sensor network is unacceptable; even the lower 
bound on the minimum achievable expected distortion does not decrease to zero with the 
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increasing number of nodes. For practically meaningful total power values, including the 
cases of P{N) = NP^^^ and P{N) = Pt^t, the lower bound on the minimum achievable 
expected distortion in (l33l) decays to zero at the rate of 



1 



(34) 



(logiV) 



x-l 



5 An Upper Bound on the Achievable Distortion 

5.1 Arbitrary Gaussian Random Processes 

Any distortion found by using any achievability scheme will serve as an upper bound for 
the minimum achievable expected distortion. We consider the following separation-based 
achievable scheme. First, we perform multi-terminal rate-distortion coding at all sensor nodes 
using [11, Theorem 1]. After obtaining the indices of the rate-distortion codes, we transmit 
the indices as independent messages using the amplify-and-forward method introduced in 
[12]. The distortion obtained using this scheme will be denoted as . 

We apply [11, Theorem 1], generalized to sensor nodes in [21, Theorem 1], to obtain 
an achievable rate-distortion point. 

Theorem 3 For all Gaussian random processes, if the individual rates are equal, the fol- 
lowing sum rate and distortion are achievable, 




N-1 



To 



pl{t) {E'^ + e'iy' pN{t)]dt 



(35) 



(36) 



where 





(37) 



and 




(38) 



K{To,0) 



K {To, ^) 



K{To,To) 



and S'jv = ■^^T,]'^ and /ig^-* , fi[ 



, f^N^li the eigenvalues ofll'j, 
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A proof of Theorem [3] is provided in Appendix 19.51 
We further evaluate (6') in the next lemma. 



Lemma 2 For all Gaussian random processes, we have 



(39) 



where A^^\ B^^^ and [6') are defined as 



N-l ^ ' Tn 



i=i ^ir^To V 



AT- 1 A^- r 



„ N-l „ ' Tn 
2 « ^ / N-l 



i=l N 



Pn \ ^^37^0 ) -pN{t) ] dt 



(40) 



and 



N-l 



To 5Z 

•J i=i -^Tm^o 



i rri 



i-1 ^ 



dt 



(41) 



and 



Af-l 



-E - 



k=0 



(42) 



respective/?/. 

A proof of Lemma [2] is provided in Appendix 19. 6[ Lemma [2] tells us that the expected 
distortion achieved by using the separation-based scheme is upper bounded by the sum of 
three types of distortion. The first two types of distortion, A^^'> and B^^\ have nothing to 
do with the rate and only depend on how well the samples approximate the entire random 
process. The third distortion, D^{9'), depends on the rate through variable 9' . 

Now, we determine an achievable rate for the communication channel from the sensor 
nodes to the collector node. The channel in its nature is a multiple access channel with 
potential cooperation between the transmitters. The capacity region for this channel is not 
known. We get an achievable sum rate, with identical individual rates, for this channel by 
using the idea presented in [12]. 

Theorem 4 When the total power P{N) is such that there exists an e > where 



lim P(N)N2-' > 1 



(43) 
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for any constant < k < 1, the following sum rate is achievable, 



= Ku\og{NP{N)) (44) 
where u is a positive constant independent of N, 



v = mill ( , - (45) 

when N is large enough. The individual rates of the sensor nodes are the same. Otherwise, 
the sum rate approaches a positive constant or zero as N oo. 

A proof of Theorem H] is provided in Appendix 19.71 Theorem H] shows that when the total 
power is such that fHSj) is satisfied, the achievable rate increases with A^. Furthermore, the 
achievable rate is the same as the upper bound on the achievable sum rate in fl2^ order-wise. 
Otherwise, the achievable rate is either a positive constant or decreases to zero, which will 
result in poor estimation performance at the collector node. 

The function {9') is a strictly decreasing function of 9' , thus, the inverse function 
exists, which we will denote as 9^{R). Let us define Da{R) as the composition of the two 
functions {9') and 9^{R), i.e., 

D^{R) = D^{9^{R)) (46) 

An upper bound on the minimum achievable distortion, i.e., the achievable distortion by the 
separation-based scheme described above, is 

= (Cf ) (47) 

We will perform this calculation when the underlying random process is in A. 

5.2 The Class of Gaussian Random Processes in A 

We analyze the three types of distortion in ( l39l) for Gaussian random processes in A. We 
will focus on A^^^ and B^^^ in Lemma [3], and on D^(9') in Lemma HI 

Lemma 3 For Gaussian random processes in A, we have 

A(^) = O {N"') (48) 
= O (at^^") (49) 

A proof of Lemma [3] is provided in Appendix 19.81 The result depends crucially on condition 
2 in the definition of A in Section d Note that since 1/2 < a < 1, both A^^) and B'-^^ 
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decrease to zero inverse polynomially as goes to infinity. 

It remains to calculate the functions {9') and (9') for random processes in A. To 
do so, we need some properties of {yU^^'* }^o^ which are stated in Lemmas [7] and [Eland proved 
in Appendix 19.91 Lemma [7] is of great importance, as it serves as a tool to link to 
{Afcj^Q, which is used in the derivation of the lower bound in Section HJ through the lower 
and upper bounds {A^}^q and {A^'}^q. Armed with the properties of /x^^^ , A'^ and A'^ in 
Lemmas [5], [6], [7] and [H] in Appendices 19.11 and 19.91 we can show the following lemma. First, 
we define two sequences and '^^, which are functions of A^, that satisfy 

hm — ^ = 0, lim ^u=^ (50) 

Lemma 4 For Gaussian random processes in A, for any constant < k < 1, lower and 
upper hounds for the function R^ {6') are 

i^xd^ 1 ^ (x^ - (1 - log 2)x + (1 - log 2)) i 

^<^a(^)< 2{x-lW ^ ^ ^ 

and an upper hound for the function {6') is 

<- 

for 0' G [t?^, 'dy] and N large enough. 

A proof of Lemma m is provided in Appendix 19.101 The proof of Lemma H] uses conditions 1, 
2 and 3 in Section [3l Let us define a sequence 'd^^, which is a function of A^, that satisfies 

lim — — -7-: — - = (53) 

Combining (l39ll . (HHl) . ( H9l) . ( |5T1) and ( |52l) . we have the following theorem. 

Theorem 5 For Gaussian random processes in A, for any constant < k < 1, the achiev- 
ahle distortion-rate function, Da{R), is upper hounded as 

n . d{l + K^jx - 1)) jx' - (1 - log2)x + (1 - log2))^-^ 

Da{R) < w^^n^-\x-iY ^ ^^^^ 

for R in the interval of 



dl (x2-(l-log2)x + (l-log2)) ..jvN-i i^^d' 



2 x-1)k^ 



(55) 
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when N is large enough. 



A proof of Theorem [5] is provided in Appendix 19. Ill This theorem shows that when R is in 
the interval (155!) . the achievable distortion-rate function is the same as the lower bound on 
the distortion-rate function in fl29l) order-wise. 



Theorem 6 For Gaussian random processes in A, when the sum power constraint satisfies 
IPD and 

lim , ^ = (56) 

an upper bound on the minimum achievable expected distortion, or equivalently, the achiev- 
able rate in the separation-based scheme, is 

= {C^ (57) 

< d{l + n^ix- 1)) jx' - (1 - log2)x + (1 - log 2))-^ / 1 \ . . 

Tofi;3^+i2^-i(a; - l)^z/^-i \\og{NP{N))J ^ ' 

when N is large enough. 

A proof of Theorem [H] is provided in Appendix 19.121 Theorem [H] implies that, when the 

sum power constraint satisfies fH3|) and fl56|) . an upper bound on the minimum achievable 
expected distortion is 



For the interesting cases of P{N) = A^Pind and P{N) = Ptot, the upper bound on the 
minimum achievable expected distortion decays to zero at the rate of 

(60) 



(log AT) 



x-l 



When the sum power constraint is such that fH3|) is not satisfied, an upper bound on the 
minimum achievable expected distortion is 0(1). 
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6 Comparison of the Lower and Upper Bounds for 
Gaussian Random Processes in A 



6.1 Order- wise Comparison of Lower and Upper Bounds 

In this section, we compare the lower bound in fl33l) and the upper bound in fl59l) . When the 
total power is large, i.e., P{N) is so large that fl56l) is not satisfied, our methods in finding 
the upper bound do not apply. Even though our lower bound in ( l33l) is valid, we have not 
shown whether the lower and upper bounds meet. However, in this case, P{N) is larger than 



e 



. / 7 2a-l fi 
jy™'°l,27'2(2,-l)'x+T+l 



^ , and this region of total power is not of practical interest. 

When the total power is medium, i.e., P{N) is in the wide range of A^^^/^+^ to 

mini' ^ 2a-l P \ 

, our lower and upper bounds do meet and the minimum achievable 

expected distortion is 

D"" = q( i .\ (61) 

The order-optimal achievability scheme is a separation-based scheme, which uses distributed 
rate-distortion coding as described in [11] and optimal single-user channel coding with 
amplify-and-forward method as described in [12]. In fact, when the total power is medium, 
as shown in (129|) and (IMIl . lower and upper bounds on the distortion-rate function, Dp{R) 
and Da{R) coincide order- wise. In addition, as shown in (12^ and dB]), the lower and upper 
bounds on the achievable sum rate, and C^, coincide order- wise as well. The practically 
interesting cases of P{N) = NPi^^ and P{N) = Ptot fall into this region of medium total 
power. In both of these cases, the minimum achievable expected distortion decreases to zero 
at the rate of 

.L-i (62) 
(log AT) 

Hence, the total power P{N) = Ptot performs as well as P{N) = A^Pmd "order- wise" , and 
therefore, in practice we may prefer to choose P{N) = Ptot- In fact, we can decrease the 
total power to P(iV) = g^n^ ^j^g 

minimum achievable distortion will still decrease to 

zero at the rate in (162|) . 

When the total power is small, i.e., P{N) ranges from A^~^ to A^~^/^, our lower and upper 



bounds do not meet. Our lower bound in (13311 decreases to zero as ^^y^^j^-jx-i but our upper 
bound is a non-zero constant. The main discrepancy between our lower and upper bounds 
comes from the gap between the lower and upper bounds on the sum capacities, and , 
for a cooperative multiple access channel. In fact, when the total power is small, as shown 
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in (129|) and (!54l) . lower and upper bounds on the distortion-rate function, Dp{R) and Da{R) 
still coincide order- wise. This total power region should be of practical interest, because in 
this region, the sum power constraint is quite low, and yet the lower bound on the distortion 
is of the same order as one would obtain with any P{N) which increases polynomially with 
A^. Hence, from the lower bound, it seems that this region potentially has good performance. 
However, our separation-based upper bound does not meet the lower bound, and whether 
the lower bound can be achieved remains an open problem. 

When the total power is very small, i.e., P{N) is less than A^~^, our lower and upper 
bounds meet and the minimum achievable expected distortion is a constant that does not 
decrease to zero with increasing N. This case is not of practical interest because of the 
unacceptable distortion. 

In the case of Gauss-Markov random process, we have x = 2 and a = (5 = T = 'j = 
1. Inserting these values into the above results, we see that in the medium total power 
region, i.e., P{N) is in the wide range minimum achievable expected 

distortion is 



For the Gauss-Markov random process, in the cases of P{N) = A^Pmd and P{N) = Ptot, the 
minimum achievable expected distortion decreases to zero at the rate of 



The conclusions in f l63p and fl64l) were derived in [1] under a different channel assumption. 
For the channel assumption in [1], the order-optimal achievability scheme was determined to 
be a decode-and-forward based scheme. The range of medium power constraints was shown 
to be slightly larger in [1], i.e., P{N) in the range of N~-^''^^^ to ^ , and this is because it 
was specifically derived for the Gauss-Markov process, instead of general Gaussian random 
processes as in this work. 

6.2 Comparison of the Constants in the Lower and Upper Bounds 

Though the lower and upper bounds meet order-wise in a wide range of total power con- 
straints, the constants in front of them are different and we aim to compare these constants 
for various total power constraints in this section. 

Combining ( ISTl) and (!58l) . when P{N) satisfies (l43l) and (|56l) . the minimum distortion 
satisfies 




(63) 
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x-1 



N 



d{l + k''{x - 1)) (x^ - (1 - log2)x + (1 - log 2))""' / 1 V"^ 
To/€3^'+i2^-i(x - \\og{NP{N)) J 



777^ (65) 



Note that k can be made as close to 1 as possible for large enough N . Let 7r(a;, v) be the 
ratio of the constant in the lower bound and the constant in the upper bound when N is 
large enough. Then, 



7r(x, v) = (2uY~^ ( ) (66) 

^ ' ^ ^ ^ Va;^-(l-log2)a;+(l-log2)y ^ ^ 

Here, x is a parameter of the underlying Gaussian random process and u depends on the 
total power constraint of the sensor nodes, P{N). It is straightforward to see that since from 
( H5|) . u < 1/4, 7r(x, u) is a monotonically decreasing function of x for a fixed u. Hence, we 
conclude that the constants in front of the lower and upper bounds differ more as x gets 
large. Since x is an indication of how much randomness the random process contains, this 
means that the more random the random process, the more the constants in the lower and 
upper bounds meet. For a fixed underlying random process, i.e., for a fixed x, 7r(x, z/) is 
a decreasing function of u. This means that the less the total power we have, the more 
different the constants will be. 

In the Gauss-Markov random process, x = 2. When P{N) = A^Pind and P{N) = Ptot, 
the ratio of the two constants is 

^(2, 1/4) = ^ 0.2708 (67) 

When P{N) = N~'^, < < |, the ratio of the two constants is 

/ 1 1 1 \ /I 1 1 \ 4 

For example, when P{N) = N~'^^^, the ratio of the constants is 

TT (2, 1/8) = i7r(2, 1/4) ~ 0.1354 (69) 



2 ^ \ 
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7 Further Remarks 



We have shown that the minimum achievable expected distortion behaves order-wise as 

Due to the order-optimahty of separation, this result can be generalized straightforwardly 
to several other scenarios. 

The result in f l7Up still holds when we allow the collector node to use its transmit antenna 
with an arbitrary power constraint. The collector node, using its transmit antenna, can send 
some form of feedback to the sensor nodes. However, the lower bound on the minimum 
distortion remains unchanged in this case, because in deriving our lower bound, we assumed 
that all sensor nodes know the entire random process, thus, forming a point-to-point system. 
In a point-to-point system, feedback, perfect or not, does not change the capacity. Mean- 
while, our upper bound is still valid, as in this achievable scheme, we choose not to utilize 
the feedback link. Hence, our result in fITOl) remains valid. For similar reasons, our result 
in ( ffOj) remains valid, when we consider a sum power constraint P{N), instead of individual 
identical power constraints of P{N)/N for all sensors. 

The result in flTOl) still holds when we allow K channel uses per realization of the random 
process, where is a constant independent of A^. This is because both lower and upper 
bounds are derived using separation-based schemes. The minimum achievable distortion 
still behaves as (ITOll . and the number K will only effect the constant in front. Due to the 
same reasoning, the minimum achievable distortion behaves as ( 1701) when we allow multiple 
transmit and receive antennas at each node, as long as the number of antennas on each node 
is a constant, independent of N. 



8 Conclusions 

In this paper, we investigated the performance of dense sensor networks by studying the joint 
source-channel coding problem. We provided lower and upper bounds for the minimum 
achievable expected distortion when the underlying random process is Gaussian. When 
the random process satisfies some general conditions, we evaluated the lower and upper 
bounds explicitly, and showed that they are both of order (-|og(-jvp(jv)))'"-i ^ wide range of 

_1 • 2(x-l) • x + T + 1 ) 

total power ranging from iV~2+'= to . In the most interesting cases when 

the total power is a constant or grows linearly with N, the minimum achievable expected 
distortion decreases to zero at the rate of (j^^^^^x-i ■ For random processes that satisfy these 
general conditions, under these power constraints, we have found that an order-optimal 
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scheme is a separation-based scheme, that is composed of distributed rate-distortion coding 
[11] and amphfy-and- forward channel coding [12]. 

9 Appendix 

9.1 Some properties of A'^ and A'^' 

In this subsection, we provide two lemmas which characterize some properties of {A'^}^q 
and {A'^j'I^Q, defined in (fTOl) and ( fTTl) . which will be useful in deriving our main results. 

Lemma 5 For any constant < k, < 1, we have 



k= 




(71) 



and 




(72) 



when 6 is small enough. 



Lemma 6 For any constant < k < 1, we have 



k= 




(73) 



and 




log 2 + a; 



) 



d-^e 



(74) 



when 6 is small enough. 



21 



9.1.1 Proof of Lemma [5] 

We will first prove (ffTl) . 



oo 

E 


K — 


oo 

E 


Ox J 


k= 


e X 




= d 


oo 

E 



(k + ciY 



4+1 

9 I 



> 



d 



X — 1 



- x-1) 



where (1751) is true when 9 is small enough, more specifically, when 
have ( 1771) because of the inequality 



r-Ci + 1 



oo _ 
k=n 



> / -dy 

'n y 



1 



{x — l)n^ 



x-1 



(75) 

(76) 

(77) 

(78) 
> Kq. We 

(79) 



and (175]) is true when 9 is small enough, i.e., for any < k < 1, there exists a 9o{k) > such 
that when <9 <9o{k), (jTS]) is true. 
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Next, we will prove (1721) . 



k=0 



9 



Ko 



k=0 



Ko 



d X 

ex 



k=Ko+l 



d 



{k + cife 



k=0 



d X 
1 

8 X 



k=0 



Ko 



k=0 



k=Ko+ci+l 



Ko , 

EiN 

fc=0 



^0) 



^1) 



d\ 1 
9 +2 



Q- Ko] log 



ex 



- - log JJ /c 

fc=K'o+q+l 

(82) 



di 



- c« + 1 I log ( - 1 - - log 



X 



! ) +-log((iro + Q)!) 



(83) 



1 

>- 
-2 



Q + 1 I log ( ^ 



X 

2 



+ 2 



+ 2 



di 



6 X 



X 



24 



Ko ^ 



>- 



X 



k=0 

d^ 



^)+|log((i^o + Q)!)-|log(27r) 



9 X 



+ |(-Q + ^)log 



d' 



X 



24 



da: 



+ C3 



(84) 
(85) 

(86) 



where (IHOj) is true when ^ is small enough, more specifically, when 
follows by using Stirling's approximation. 



Cl 



n\ < ^27™"+ 2 e "+12" 



^7) 
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(!85|l follows because C3 is a constant, independent of 6', defined as 

C3 = E 2 i + f ((^0 + q)!) - I log(2vr) (88) 

fc=0 ^ ^ 

and fISB]) is true when ^ is small enough, i.e., for any < k < 1, there exists a 6i{k) > such 
that when <9 <9i{k), is true. 

Therefore, for any < k < 1, (17T!) and (!72|) hold when 6' is small enough. 

9.1.2 Proof of Lemma [6] 

We will first prove (1731) . 



E 



E 



+Cu +1 



' +C„ +1 



00 , 

Ed 

.=[(f)*J+l 

d 



< 



(x-1) 

d^ 
{x — l)/t 



x-l 



where f l89p follows when 9 is small enough, more specifically, when (|) + 
In obtaining (!9T|) we used 



CXD _ 



r'CxD 

< / -dy 



k=n 



k"" Jn-iV''' (x-l)(n-l) 



(89) 

(90) 

(91) 

(92) 
+ 1 > i^o- 

(93) 



and fl92p follows when 9 is small enough, i.e., for any < k < 1, there exists a 92{k,) > 
such that when < ^ < ^2(fi:), (I92]) is true. 
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Next, we will prove (1741) . 

fc=0 ^ ^ 

1(1)"+^" I 

fc=0 ^ ^ fc=ii'o+l "7 / 
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d\- 1 a; 
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log 2 + X \ X - 

- Tlog 
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where flMj) is true when 6 is small enough, more specifically, when 
have (1951) because 



d 



(k - cuYe 



> 1 



> Kn. We 



(103) 



for all k between Kq + 1 and 



_L 

(I) + c„ , and when 9 is small enough such that 



e<\k, k = l,2,---,Ko 



(104) 



We have (1971) because we defined 



Ko 



Cl 



k=l 

We used Stirling's approximation 



2"° d ^ 2 ^ (k- CuY 

k=\ fc=c„+l ^ ^' 



(105) 



n! > V27rn"+2e "+i2n+i 



(106) 



to obtain (1^ . and (110 II) follows by using 



log(l + x) < X, X > 



(107) 



( 11021) follows when Q is small enough, i.e., for any < k < 1, there exists a ^3(k) > such 
that when <Q^{k), (fT02D is true. 

Therefore, for any < /t < 1, (173!) and (!74|) hold when 6' is small enough. 

9.2 Proof of Lemma [1] 

For any < < 1, when Q is small enough, the results of Lemma [5] hold. 
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From (l22l) . we have 



fc=0 ^ 

Kxd^ 1 



(108) 
(109) 

(110) 
(111) 



where in (11091) we have used the definition of sequence A'^ in (fTOl) and the observation in ( JT2l) . 

(1 11 01) follows when ^ is small enough, more specifically, when 9 < Xkq and ^ — c; > Kq. 
Ls^ J 

(11 111) follows from (1721) in Lemma El 
From (l23il . we have 

oo 

Z}(0) = To-i5^min(^, A^) 

fc=0 

oo 

>To-^5^min(^,A',) 



fc=0 

1 

d X 



A;=0 



1 

— -Cl + 1 

e X 







- Q + 1 j 


( 






9 X 





n-l gl- 



X - 1 / Tf 



(112) 
(113) 

(114) 

(115) 
(116) 



where in (11131) we have used the definition of sequence A^ in (ITOl) and the observation in ( fT2l) . 
(11141) follows when ^ is small enough, more specifically, when 9 < Xkq and ^ — q + 1 > 

\_6x 

Kq. (11151) follows from (I7T1) in Lemma [5l (11161) is true for small enough 9, i.e., for any 
< K < 1, there exists a ^4(k) > such that when < 6^ < ^^4(fi;), (I116P is true. 
Therefore, for any < k < 1, fl271) and fl28|) hold when 6' is small enough. 
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9.3 Proof of Theorem [T] 



R{0) is a strictly decreasing function when ^ < Ai. Hence, when 6 < Xi, the inverse function 
0{R) exists. For any < k < 1, when 6 is small enough, or equivalently, when R is large 
enough, from fl27|) in Lemma [1], we have 

e{R) > d (^^y R-^ (117) 

Using (11171) and fl28l) . for any < k < 1, fl29l) holds when R is large enough, since D{9) is a 
nondecreasing function of 6. 



9.4 Proof of Theorem [2] 

When P(A^) is such that (l30l) is satisfied, from fl25|) . we see that in this case increases 
monotonically in A^. Hence, when is large enough, will be large enough such that 
Theorem [1] holds. Hence, for any constant < k < 1, a lower bound on the minimum 
achievable expected distortion is 

Df" = D.iCH) (118) 



x-1 



> ^1 , _J (.,)^-. ^^^^^^J (121) 

where (11201) follows from fl25|) . and the last step follows when is large enough, i.e., there 
exists an No{k) > 0, such that when > Nq{k), (I12ip is true. 

Therefore, when P{N) is such that fl5Ul) is satisfied, for any < k < 1, fl5T]) is true when 
A^ is large enough. 



9.5 Proof of Theorem [3] 

We restate the generalization of [11, Theorem 1], which appeared in [21, Theorem 1] for A^ 
sensor nodes below. This provides us with an achievable rate-distortion point. 

Theorem 7 [11, 21] If the individual rates are equal, a rate- distortion sum rate Rc and 
distortion are achievable if there exist random variables Ti,T2, ■ ■ ■ ,T/v with 

(Sit), T{i|c) ^ S,: ^ r„ 2 = 1, 2, ■ ■ ■ , AT (122) 
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and an estimator function 

S{t)^g{T,,T2,--- ,Tr,) 



(123) 



such that 

Rc>I{Si,S2,---,SN;T,,T2,---,Tr,) (124) 
D,>E[d{S{t),g{T,,T2,---,TN))] (125) 

Wc obtain an achievable rate-distortion point when we specify the relationship between 

{s{t),{s,}z„mr=i)^ 

Ti^Si + Wi, i = l,2,---,7V (126) 

where Wi, i — 1,2, ■ ■ ■ ,N, are i.i.d. Gaussian random variables with zero-mean and variance 
aj) and independent of everything else. Here, we can adjust to achieve various feasible 
rate-distortion points [11]. 

We choose the MMSE estimator to estimate S{t) from observations {2^};^^. Hence, the 
achieved distortion is 

D^i^i) - Yo r " ^""^^^ ^^"^ ^ ^""^^^^ ^^^^^ 

The sum rate required to achieve this distortion is 

i'^o) ~ H^ii S2, - ■ ■ , Sn', 7i, T2, ■ • • , Tjv) 

= ^logdet(^/+^Sjv^ (128) 

where iJ,'^\ ■ ■ ■ , /i^^ are the eigenvalues of Hn- 

Next, let e' = ^al, = and /x^^^' = ^fJ^i^^- We define two functions of 9' 

as 

N-l / (AT)' \ 

fe=0 \ / 
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and 

= D^{al) = 1 {K{t, t) - ^P^(t) {^'n + O'ly' pN{t)^ dt (131) 

and by definition, sum rate {6') and distortion [9') are achievable for an arbitrary 
Gaussian random processes. 



9.6 Proof of Lemma [2] 

Using tlie matrix inversion lemma [22], 



'-1 

AT 



we have 



where we have defined 

l-To 



N -I 

A^— 1 n _i_7^0 / \ — 1 

+ E rr ° ^rwsiv' + sivM ^N^^^m (138) 



(132) 



1 /-To /I \ -1 

+ p?^ws;v"'(^^^+s;v"'j s;,-W(t)rft (133) 

=L'f) + D(^)(^') (134) 



Df^ =^ ^ " (i^(t, t) - ^P^(t)S;v" Viv(t)) rft (135) 

^^"'H^o p^ws;-^(^^/+s;-^j s;v"V^(t)rft (136) 

We continue evaluating D^^\0')^ 



^'n' ( PN ( ^rA^o ) - A,(t) ) (137) 
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where Aj(t) is defined as 

AJi) = Hat I 

^A^- 1 

for j^Tq <t< jj^Tq, and fll38p follows based on the fact that 



Mt) = PN iir^n ) - PNit) (139) 



pI ( J:'^^ = (140) 



where is the row vector whose i-th entry is 1 and all other entries are 0. 
The eigenvalues of ^JjJ + S^"*^ j Sj^^ are 



k = 0,lr-- ,N -1 (141) 



which are smaller than the corresponding eigenvalues of S»f , i.e., -jw- Thus, the third 
term in fll38p is bounded by 

i=l -J jTZTfTo V / 

^ iV^ E J^^ Ajm'^'Mt)dt (142) 



To further upper bound the third term in (11381) . we write 

r-To 



Di"^ = 4 / " (^(^' ^) - j;^PNmN'PNit)^ dt (143) 

1=1 



^'n' (^pn (^ttt^O " ^^^^0 ) ^^^^^ 
+ T E / (^^(^))^ - 1^ E / A.(t)^E;,-i A,(t)dt (145) 



1 /-iv^^o 

Mtfj:'^'Mt)dt (146) 

1 i^Tn 



i=l 7^:1-^0 
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where we have defined 



1 X — / iV-1 "J 



N-1 "'A^ 



N-l „ i Tn 



(147) 



N-l ^ i Tn 



i=l iv 



N-l ^ » Tn 



I 1— L rr, 

1 "'iv^iTo 



1 It \ To ] - PN{t) ] dt 



(148) 



Then, we have the third term in (11381) upper bounded by A^^^ because of (I142p . (I146P and 
the fact that Dif^'' is non-negative, i.e.. 



1 



N-l „ i Tn 



=1 "'jv^-'o 

Furthermore, we can see from (I146P that 



Now, we evaluate the second term in (I138p . Since, 
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|A.(t)|| 
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(149) 



(150) 



(151) 
(152) 

(153) 
(154) 



where || • II2 denotes the spectral norm of a matrix, which is defined as the largest eigenvalue 
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of a matrix [22]. Therefore, the second term in (11381) is bounded by 
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where we have defined B^'^'^ as 
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Finally, the first term in fll38p is bounded by 



■I - 1 
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(155) 

(156) 
(157) 



(158) 
(159) 



(160) 

(161) 

(162) 
(163) 



where the last step is by the definition of D^{6'). Hence, for an arbitrary Gaussian random 
process, by ffTMj) . (fT^ . ffM . (fTO . ffTFTD and ffM . we have shown that 



L>f (^0 < 2A(^) + + (^') 



(164) 



9.7 Proof of Theorem [4] 

Each round of communication will take 2Nn time slots. In the [2n{i — l)]-th to [2nz]-th time 
slots, node i transmits at rate Ri, while all other nodes act as relay nodes and transmit no 
data of their own. In the end, the achievable sum rate is — -. We will show that each 
node can achieve Ri = , and thus, all nodes can achieve the sum rate of with identical 
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individual rates. 

We will consider the transmission of the data of node i. Node i codes its message using 
capacity achieving single-user coding techniques with codeword length n. Each codeword 
symbol requires two time slots. In the first time slot, node i transmits its codeword symbol 
using power P{N). All other nodes remain silent, and receive a noisy version of node 
i's transmitted signal. The collector node ignores its received signal, which is suboptimal 
but eases calculation and does not affect the scaling law of the achievable rate. Identical 
individual power constraints of P{N) /N for the nodes are satisfied, since all nodes take turns 
and node i will do this only l/A'"-th of the time, therefore its transmit power in l/A'"-th of the 
time is P{N). In the second time slot, all sensor nodes, except node i, amplify and forward 
what they have received in the previous time slot to the collector node using an individual 
power constraint P{N)/N. The collector node, after 2n time slots, decodes using capacity 
achieving single-user decoding techniques. Now, we calculate the rate achievable with this 
scheme. In the first time slot, sensor node j receives 



Yj = hijXi + Zj, i, j = 1, 2, • • • , TV, j i 



(165) 



and in the second time slot, sensor node j transmits 



Xj — PijYj 

= pijhijXi + PijZj, i,j^l,2,---,N, j y^i 



(166) 
(167) 



where Pij is the scaling coefficient of node j when it amplifies the signal it received from node 
i. In order to satisfy the identical individual power constraints, have to satisfy 



Vi,j = l,2,---,7V 



(168) 



The collector node receives 



N 

N \ f ^ 

PijKjhjo I Xj -I- I ^ hjoPijZj j -I- Zq 



(169) 
(170) 



Therefore, the achievable rate is. 



1 + 



V 



A 



(171) 
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where we have ^ because we used two time slots to transmit one codeword symboL We 
choose 



= (hijhjo (172) 
where, in order to satisfy the power constraint, the constant ( must satisfy 

We can choose C as 

= - - (174) 

Thus, from fll7ip . an lower bound on the achievable rate is 



( c-(£-.,,.ft|ftjo)V(iv) \ 1 / i^ciN-imm . ,,,,, 



\ C=(Ef=„^.ft|AJ„) + i ) 

Clearly, rate can be achievable by any node i. We have 



C - 1 .OS I + mml^IS^] (176) 
1 / hf(P{N)fN \ 

where the last step follows when N is large enough such that ^^"'''^ > y. 
When P(A^) is such that 



for any < /t < 1, we have. 



lim = (178) 



- log 1 



h 



NPiN)] (180) 
>^ log (iVP(iV)) (181) 
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for large enough, i.e., there exists Ni{k,) > 0, such that when N > Ni{k), (I179P and (11811) 
are true. 

When P{N) is such that 

hm P{N) = I (182) 

N^oo 

and / is a number that satisfies < / < oo, fix some small /q > 0, there exists an N2{lo) > 
such that when > N2{lo), we have, 

l-lo< P{N) < / + /o (183) 

Hence, when N > iV2(/o), for any < k < 1, 

Cf > 1 log fl + p(iV)iV^ (184) 

> I log (iVP(iV)) (185) 

where the last step follows when N is large enough, i.e., when there exists an N^in) > 0, 
such that when > max {N2{lo), N^^k)), (I185p is true. 
When P{N) is such that 

lim P{N) = (186) 

N~>oo 

and there exists a constant < e < |, such that 



lim P{N)N^-' > 1 (187) 

W— too 



we have, for < /t < 1, 

C>^fog(i + ^(niv)fivj (1^ 

>^log{{PiN)YN) (189) 

= ^ log(Arp(iV)) + ^ log(P(Ar)) (190) 

>^^Jog{NP{N)) (191) 

where the last step follows from 

f (l - ^og{NP{N)) + ^ \og{P{N)) = l^^\og{P{N)N^-^) > (192) 
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when is large enough, i.e., there exists an A^4(/t) > 0, such that when N > N4{k), fllSSp . 
(11891) and (I192p are true, and therefore, fllQip is true. 

Thus, combining all possible cases of P{N), we see that when P{N) is such that there 
exists a constant e > 0, such that 



lim P(N)N^-' > 1 



for any < k < 1, the following rate is achievable, 

= Ku\ogiNP{N)) 

where constant u is 

u = min 



(193) 



(194) 



1 + 2e' 4 



(195) 



when is large enough. 

Since all nodes take turns applying the same scheme, the individual rates of all sensors 
are the same, and the achievable sum rate is fll94p . 

For all other P{N), from fll77p . we see that the achievable sum rate approaches a positive 
constant or zero as goes to infinity. 



9.8 Proof of Lemma [3] 

We first consider A^^\ 



1=1 >' TTTi'^O 
^ 1=1 ^ TT^ 



N -1 N -1 



dt 



Pn 



I - 1 



N -I 



To - PN{t) dt 



N-l 



<- 



„ N-l „ ' Tn 



To 

i=l "^jv^T^o 
N~l „ ' Tn 

<^ V / B 



To, T7 7 To 



pN 



N-l 'N-l 
i-1 



dt 



N-l 



-T 



PNit) 



dt 



V2 



N-l 



N-l ^ ' Tn 



(196) 



B 



i=l N 



N-l 



To dt 



1 



(197) 

(198) 

(199) 
(200) 
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where ( 11981) follows from condition 2 in Section [31 Using similar ideas, we have 



„ N-l „ ' Tn 



-y 

To ^ /-I 



1=1 iV 



^^0 



i - 1 ^ 



2 - - / iv 



< 



U i=\ -^Trrr^o \m=0 

2 » — / iv-i " 



N-l 



25t; 



iV5 



^■m=0 



5 



PN{t) 

N-l'"' N-l 

To 
N-l 



dt 



K t 



mTo 
N-l 



1 

2\ 2 



dt 



{N-iy 



Q(N^^ 



{NY 



1 

2\ 2 



dt 



(201) 
(202) 

(203) 

(204) 
(205) 



9.9 Some properties of fi]^ 

Lemma 7 For all Gaussian random processes in A, let Ki{N) be a sequence of numbers 
that satisfies 







lim 

N^ocKi{N) 

{K,{N)+B, 

lim ; r 

N^oo {N - l)'i' 

Ki{NY+^+^ 



\2r 







(206) 
(207) 
(208) 



Then, for each k such that k < Ki{N), there exists an eigenvalue /i'*-^-*, different for each k, 
of S'^ such that 



(209) 



for some di> Q and some positive integer Bj, both independent of k and N , when N is large 
enough. 

Lemma [7] shows that the convergence of jjl]^'^ to is not uniform, and the approximation of 
using Afc is accurate only when k « Nr and >> dij^^^. When the conditions 
of Lemma [7] are satisfied, we label the /i'^^^ that satisfies (12091) to be /i^^^ for k < Ki{N). 
The remaining — Ki{N) eigenvalues of yu'(A^) will be labelled according to the order from 
large to small. 
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^ = 0, nm_ < = (210) 



Lemma 8 For all Gaussian random processes in A, define two sequences 'd^ and as 

1 

For any constant Q < n < 1, we have 

N-l ,1 



E s j-^^-i (211) 



(a; — 1)k 

' +Cu I +1 



when 6' E [^l^'^u] ^'^^ ^ ^■^ large enough. 

Lemma [HI shows that the sum of the eigenvalues that do not converge to for k = 
0, 1, ■ ■ • , (^) ^ + is quite small. 

9.9.1 Proof of Lemma [7] 

By definition, for any k satisfies 

h(pk (l^To] = [ \ (l^To, s] Ms)ds, V/ = 1, 2, ■ ■ ■ , iV (212) 



^A^-1 7 Jq \N-1 
We rewrite the right hand side of fl212p by 

^° ^"'^^~^n^^To]<pJi^To]+e%(^To] V/ = 1, 2, ■ ■ ■ , iV 



iv-i^^Viv-r^'iv-rv""^ Viv-rv ' \ n-i 

i=i 



(213) 



where [jjA^Tq) is defined as 



i'^°/..//-i„ \ //-i„ ?-i„\, 



To ,V/-1 



Using (I2T2D and (l2T3|) . we have for any / = 1, 2, ■ ■ ■ , iV, 

(215) 
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i.e., we have 



N -1^ \N -1 



N 



N 



i=l 



To / ^-1 



(216) 
(217) 



Let us define vector a^;'^'' of lengtli of by defining its l-th. element to be ^J 

and vector b^^^ of length of by defining its Z-th element to be J ■j^zj't'k {j^Tq), we have 



in matrix form 



Afcb 



(218) 



The links between the eigenvalues of and the eigenvalues of K{t,s), i.e., the A^s, will 
be determined using (12181) . To do this, we first bound three quantities. 



AN) 



AN) 



for k,m,l < Ki{N) and m ^ I. 



We first upper bound |0jfc(To)|. Let Fk{s) be defined as 



(219) 



Then, by the mean value theorem on interval [0, Tq], we have that there exists a T' G [0, To] 
such that 



Hence, using condition 3 in Section [3l we have 



^k{t) - MT')\ < B,{k + B.YTl t e [o,To; 



(220) 



(221) 



Thus, 



M<B,{k + B,yT^ + l, tG [0,To] 



(222) 
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Now, we analyze the norm of a^^-*. From the definition of e% [^-^Tq) in (EHj), we have 
/ - 1 



N-1 



-Tn 



N-1 „ ' Tn 
i=l -J 

To 



N -I 



N-l ^'N-l 



N -I 



ds 



+ 



A^- 1 

Til 
^ 



K ( ^To,ro 

iV- 1 °' ° 

+ 



(iV- 



N-l 



S -D2-to 



(Ar-l)/3 



+ 



iV- 1 



S -D2-io TTT 7T^ + 



< 



[N-iy {N-iy 

B^Tl^^ + To'+^i?(To)53) {k + max(5i, 54))" + Toi?(To) 



(AT - l)/3 



l/r\- 



< 



< 



max(l, (52Tj+^ + To^+^i?(ro)53)) ((fc + max(Bi, 54))" + ((Toif (Tq)) 

(iV- 

max (1, 21-^ max (^1, (^3X0^+^ + T^^^ K{To)B^yj (k + max(5i, ^4) + {ToK{To)) 



(223) 
(224) 
(225) 
(226) 

(227) 

(228) 
i/rY 



{N - ly 



< Be 



{k + B'^y 
{N -ly 



(229) 
(230) 



where ( 12241) follows because the random process satisfies condition 3 in Section [3], and because 
K{Tq) is defined as 



max 

KKN 



(231) 



and is a finite nonnegative number since K{t, s) satisfies condition 2 in Section [3] and thus, 
is continuous, fl225p follows from fl222p . and (12261) follows because (3 <1 from condition 3 in 
Section [31 and (12291) follows because for VM,f > 



+ < {u + vy, r > 1 

+ ^ (u + v 

< I 1 . < r < 1 



(232) 
(233) 



(12301) comes because we define the variables B^ and -B7, which are both independent of k 
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and iV, as 



^6 = max (1, 2'-^) max (l, (^B^T^^^ + T^+m{To)B, 



B' 



max{B,,B^) + {ToK{To)) 



l/r 



(234) 
(235) 



Note that B'j is a positive integer. Finally, we calculate the norm of vector a^^^ as 



AN) 



N 



To ^ / / - 1 



Tn 



, NTo {k + B',r 
^''W^l' {N-IY 
{k + B'jY 



< 2B.WTr 



(236) 

(237) 
(238) 



{N - 1)P 

where (12381) follows when is large enough, more specifically, there exists an interger A^i 
such that when > A^i, we have -i/ jt^t < 2. 



(239) 



Now, we will calculate the norm of vector b^,^''. We write 



where 6% is defined as 



N-l .^To 



I - 1 



Tn Us - 



To ,2 



iV- 1 



0^(To) 



(240) 



Using dHl), we have for any Si, S2 £ [0, Tq] 



|0fe(si) - 4>1{S2)\ = |0fc(Sl) + MS2)\ IM^l) - MS2)\ 

<2 max \(Pk{s)\B3{k + BY)^\s,-S2\^ 

se[0,To] 

< 2 {Bs{k + B,yT^ + 1) B,{k + B,y \s, - s,\ 



(241) 
(242) 

(243) 
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where ( 12431) follows from fl222p . The approximation error, 6^ satisfies 

N-l 



i=l "'j^^O 



ds + 



A^- 1 

k2l 



. ^i+, 2 (i?3(A: + B,YT^, + 1) B,{k + B,y To<l>l{To) 



(A^- 1)^ 



. ^1+^ 2 (i^alA: + i?4)"T; + 1) B^jk + i?4)- ^ Tq (i^alA: + 54)^^0-^ + 1)^ 



(A^- 1)T 



iV- 1 



. ^i+-, 2 (j?3(fc + i?4)"To^ + 1) Bsjk + ^ To (i?3(fc + i?4)"To' + 1)^ 



< 



ST^^'^Blik + B^f- + 4Tl-^''B^{k + 54)^ To 



(iV- 1)T 



+ 



(244) 

(245) 
(246) 
(247) 
(248) 



where fl246p follows from fl222p . and (12470 follows from the condition of 7 < 1 in condition 
3 in Section [31 Due to the fact that Ki{N) satisfies (I206P and (12070 . for a fixed constant B^ 
that satisfies < -B5 < 1, Then, there exists an integer A'q > 0, such that for N > Nq, 



3T^^'^Bl{K,{N) + B,f^ + AT^^^B,{K,{N) + B,Y ^ Tp ^ ^ 

+ TTT — <: -D5 



{N -lY' 

Hence, for any k < Ki{N) and N > Nq, we have 

^T^-'^^Blik + B^Y^ + AT^+^B^ik + B^y , To 



(A^- 1)^ 



(249) 



+ 



< 



(AT -1)7 {N-iy 
3T^^'' Bl{K,{N) + B,r + 4Tl^' B,{K,{N) + B.f , 



Tn 



{N -lY 



< B. 



(250) 

(251) 
(252) 



Finally, by the definition of b^^^ , we have 



AN) 



^ N-l 



i-1 
N-l 



Tr 



1-4 



> a/1- 141 



> a/1 - 5, 



(253) 

(254) 

(255) 
(256) 



where (12541) follows from (I239p . Similarly, we have 



AN) 



< y^l + B, 



(257) 
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Next, we show that based on the orthogonahty of the eigenf unctions of (f^kit), the sampled 
version b^^^s are almost orthogonal. Using (IT^ . we have 

|0m(Sl)0/(Sl) - 0m(s2)0/(s2)| (258) 
= |0m(Sl)0Ksi) - 0m(Sl)0Ks2) + 0m(Sl)0/(s2) - 0m (^2) (-52) | (259) 

< |0m(Sl)0i(si) - 0m(Si)0z(s2)| + \4>m{si)(f)l{s2) - 4>m{s2)(f)l{s2)\ (260) 

< max |0m(si)| |0i(si) - 0i(s2)| + max |0i(-52)| |0m(Sl) - 0m(s2)| (261) 

sie[0,To] S2G[0,To] 

< {B,{m + B^YT^ + 1) B,{1 + B^y \s, - s^l'' + iB,{l + B^yT^ + 1) B,im + B,y \s, - 

(262) 

= {2Bl{m + B^y{l + B^yr^ + B-i{l + 54)" + B^{m + 54)") |si - S2[' (263) 



where (12621) follows from fl222p . Let m and / be two different integers, that belong to 
{1, 2, ■ ■ ■ , A^}. Then, we have 



i - 1 



= / (i>rr^{t)mdt = J2 w^"^'- \N-i 

i=l 







i - 1 
N-1 



To]+e 



m,l 
N 



(264) 



Then, we have 



m,l 



i - 1 

'^l 1 TT 



N -l 



< 



4>m{t)4>l{t) - 4>n 



i - 1 

N-1 



-Tr 



VI 



i - 1 



Tn 



^^1+^252(771 + 54)^(/ + B^yr^ + 53(/ + B^y + 53(m + 54) 



(A^- 1)^ 

(53(m + B^yT^ + 1) (53(/ + B^yr^ + 1) 



iv- 1 



^^i+^25|(m + B^yjl + g4)"r(7 + B^{1 + E4)" + 53(m + E4) 



(AT - 1)7 

(53(m + 54)^To^ + 1) (53(/ + E4)^To^ + 1) 



(265) 



{N -ly 

SBlim + B^y{l + B^yr^^^^ + 2B^{1 + B^yT^-^"' + 2S3(m + 54)^tJ+^ + To 



(266) 
(267) 
(268) 
(269) 
(270) 



{N -ly 



(271) 
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For m, I < Ki{N), we have 



_in,l 
'N 



< 



< 



1+7 



(iV- 1)T 



(272) 
(273) 



where f l273p follows when is large enough due to the fact that Ki{N) satisfies (12061) . i.e., 
there exists an integer N2 such that when > N2, (I273P is true. The right hand side of 
(12731) converges to zero as N goes to infinity due to the fact that Ki{N) satisfies (12071) . We 
have 



N 

E 

i=l 



Tr 







N-1 



i - 1 
N-1 



n 



Vi 



i - 1 
N-1 



Tn 



_m,l 
'N 



< 



ABl{K^{N) + B^f-T^-^^^ 
(iV-l)T 



(274) 
(275) 



which means that vectors b^,^^'' and bl^"* become more orthogonal as gets larger. 

Now, we are ready to establish the link between the eigenvalues of and A^. From 
fl2T8l) . we have 



Hence, 



bf ^ = (S^ - A./)- (-af )) 



(276) 



Thus, we have 



< 



(N) 



mm 

0<m<Af 



AN) 



mm 

0<m<Af 



- A.) < 



AN) 



AN) 



< 



O D /rp~(k+Bj)^ 
^-D6V Jo (Ar_i)/3 

{k + B',Y 



{N-iy 



(277) 
(278) 



(279) 

(280) 
(281) 
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where ( 12811) follows by defining do as 



d« = (282) 
Hence, for /c = 0, 1, 2, ■ ■ ■ , Ki{N), there exists an eigenvalue fj,'^^^ of such that 

|a.'™-A.|<4{^^ (283) 

when is large enough, more specifically, when N > max(Ai"o, A'^i, A^2)- 

For A; = 0, 1, 2, ■ ■ ■ , Ki{N), if we label the /i'(^) that satisfies (12551) to be /i^^^', then when 
Afc for different ks are sufficiently close, more specifically. 



\\rn-\i\<2do^-^^^^p-^-^, m,l<K,{N),mj^l (284) 
fim'^ and /i;^'' might correspond to the same eigenvalue of S'^, which is undesirable. If we 

(k+B'Y 

relax the minimum distance of do (jv_i)ii ; "we will be able to eliminate this problem. Thus, 
we will next show that for k = 0, 1,2, ■ ■ ■ ,i^i(A^), there exists an eigenvalue fi'^'^^ of S^, 
different for each k, such that 

- A,| < (2x + ^)Vd'.do^^^^ (285) 

when is large enough, where we define x = niax(_K'o + 1 + c„ + q , 2c„ + 2q + 1) and constant 
d2 as the largest root of the following second-order equation 

(1 - B,)dl - 2 ((1 - B,) + 3x(l + B,)) d2 + {l- B,) + 2x(l + ^5) = (286) 

It can be checked that both roots of the above equation are real, and the largest root is a 
positive constant, strictly larger than '^^'^^^^''^ + 1, that is a function of x ^i-iid B^. 

First, let us define a cluster of As. We say that x -^s are a cluster, where with no loss of 
generality, we may label these As A^, A^+i, ■ ■ ■ , Afc+;^_i, if 

XkM - Afe+i+i < 2v^rfo ^^^y_^^^/^ , / = 0, 1, ■ ■ ■ , X - 1 (287) 

Note here that whether the As are in a cluster depends on A^. 

Next, we prove that the number of As within a cluster is upper bounded by x when is 
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large enough. For k > Kq, we have 



d ^ d 

<\k< T, (2^ 



{k + CiY - {k- CuY 

d d 

< Afc+c„+q+l < 71— — TTT (289) 



{k + 2q + Cu + ly - - {k + ci + ly 

Hence, for every k > Kq, the distance between and Xk+cu+ci+i satisfies 

A. - A,+,„+,,+i > - (^^^^ (290) 

which is a non- increasing function of k. Thus, for all Kq < k < Ki{N), the distance between 
Afc and Xk+cu+ci+i satisfies 

d d 

Xk - Afc+,„+,^+i > ^^-^^^^^--^ - (291) 



d f f 1 



(292) 



- (iri(iV) + Q)H ^i(A^) + Q + l 2 (iri(iV) + Q + l)2; ^ ^ 

_ •^'^ -^(-^ ~ -'-)'^ (294) 



(iri(iV) + Q)^+i 2{K^{N)+ci\ 



x+2 



where fl295p is true when is large enough due to the fact that Ki{N) satisfies (I206p . i.e., 
there exists an integer N'^, such that when N > N^, (12951) is true, and (12961) is true when 
is large enough, due to the fact that Ki{N) satisfies (12081) . i.e., there exists an integer A^4, 
such that when N > N4, fl296|) is true. 

Hence, for all Kq < k < Ki{N), when N is large enough, more specifically, when > 
max^Ns, N4), due to the sufficient distance between and Xk+c^+ci+i, shown in (I296p . they 
cannot be in the same cluster. Hence, we may conclude that for large enough N, the size of 
a cluster is at most which is a finite number. 

Following from (I218p . we have 

(A,J - E^) bf ) = af ) (297) 
Let the eigenvalues and the corresponding eigenvectors of S'^ be fi\^^ and u-^-*, i = 
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1, 2, ■ ■ ■ , iV, with arbitrary labelling of the eigenvalues and eigenvectors. Then we have 



N 



i=l 



AN) 



(298) 



We take the norm squared on both sides, and due to the orthogonality of eigenvectors 
we have 



(TV) 
i 5 



TV 



(TV)' 



2=1 

and we also have 



AN) 



k = 0,1,2, 



(299) 



N 



EH 



.W^, (TV) 



AN) 



, A; = 0,l,2,--- 



(300) 



i=l 



Let Xk, Xk+i, ■ ■ ■ , Xk+x-i be a cluster, and from previous arguments, we know x ^ X- 
Furthermore, we are only interested in the first Ki{N) + 1 eigenvalues, and therefore k + 
X — 1 < Ki{N). We will prove by contradiction. Suppose that only ^ number of /i-^-* s are 
within distance 



d2do 



{k + x + B',y 

{N -If 



(301) 



from any of the A^, A^+i, ■ ■ ■ , Afc+^_i, with 1 < <j < x, we will show that there is a contra- 
diction, and therefore, we can conclude that our assumption that <j < x number of ^f^^ s 
are within distance (13011) from any of the A^, A^+i, ■ ■ ■ , \k+x is not correct. Note that ^ > 1 
because we have already proved fl283p . Based on fl28ip . the distance in fl30ip satisfies 



d2do- 



{k + x + B',y 



{N-1)P 
Then, based on fl299p . we have 



> 



\"k+l\ 



/ = o,i,--- ,x-i 



(302) 



(Vd~2\ 


\'^k+i\ 






|b 


{N)i 
k+l\ 





TV 



E " 



AN)T,iN) 
"k+l 



< 



i=?+l 



AN) 
*-k+l 



/ = o,i,--- ,x-i 



(303) 



where we have labelled the /x''-^-' that are within distance (130 ip from any of the A^, A 



k+l, 



\fc+X 
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, /ig^'' , ■ ■ ■ , ni^'^ . Hence, we have 



E (-^bKl) / = 0,l,...,x-l (304) 



i=?+l 

Together with (13001) . we have 



Z (".'"'^Ci) a ' , " ■ ' = °.1.--A-- 1 (306) 

1=1 

Since the u^^^"* form a complete set of basis in M^, we can write b^^] as 

bS = E"^+MuS''^+vS, / = 0,l,---,x-l (306) 



i=l 



where v[.^] is orthogonal to u^-^-*, for i = 1,2, ■■ ■ , If we take the expression of b^^] in 
(13061) and plug it in (I305p . we get 

E(W>^^^^^7^, / = 0,l,2,...,x-l (307) 

1=1 

From (13061) . we get 

I Ib^l I P = E {a,^i,r + 1 1 vKl I / = 0, 1, 1 (308) 



Hence, we conclude that 



i=l 



||uW||2 



IK;;il'<^t^, / = 0,l,---,^-l (309) 

Furthermore, from (I306p . we have 

bl+!^bi+l = E"fc+™.i"fc+M + vi+!lv^^j, m,/ = 0,l,--- ,^-l,m^/ (310) 
1=1 

Hence, we have 

E (^k+m,iak+i,i = hi%hf^] - vi^^ill, m, / = 0, 1, ■ ■ ■ , ^ - 1, m 7^ / (311) 

i=l 
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and for m, / = 0, 1, ■ ■ ■ , <^ — 1, m 7^ /, we have 



Oik+m,iOik+l,: 



i=l 



< 



< 



< 



< 



< 



+ 



^k+m ^k+l 



2Trril+2j 



{N-iy/ 



+ 



+ 



k+m 
-'k+m 



ABl{K,{N) + B.fX:^ 1 + ^5 



'k+l 



"k+l 



do 



2(1 + ^5) 
do 



(312) 
(313) 

(314) 
(315) 
(316) 



where fl3T3|) follows from (12751) when N > N2, (1314]) follows from (l309|) . (I3T5D follows from 
(12571) when N > Nq, and (13161) follows when N is large enough, due to the fact that Ki{N) 
satisfies (I207p . i.e., there exists an integer Nq, when N > Nq, ( 13161) is true. Let us define 
matrix A to be 



A 



Oik+1,1 Oik+1,2 



(317) 



and define vectors b, v, u to be 



"k "k+<; 
"k+l "k+<; 


, V = 


^k "k+<; 
^k+l "k+<; 


, u = 


"1 "k+i 
"2 "k+i 


b(^)^ b^"^) 
."k+ci-l^k+c;. 











(318) 



Then, by ( 13061) . we have 



h = Au + v 



(319) 



In other words, 



u = A-'^ (b - v) 



(320) 
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thus, we have 



u|r=||A-^||'(||b|| + ||v||)^ (321) 



We start by evaluating \\A ^\\^, which is equal to the inverse of the smallest eigenvalue of 
A^A. From the definition of matrix A in fl317p . we have 

A^A = D + E (322) 

where D is an <j x <j diagonal matrix with the /-th diagonal element being Yl\=i (c^fc+i-i.i)^, and 
is an ^ X matrix with zero diagonals and (m, /)-th element being Yl\=i (^k+m-i,iC(k+i-i,i, 
when m I. The absolute difference between the smallest eigenvalue of A^A and D is upper 
bounded by \\E\\^ [23]. The smallest eigenvalue of D is 



12 

[ak+i,i> > mm (323) 

> - 'X' - (324) 

0-2 

where ( l323ll follows from (13071) . and (l324ll follows from (12571) when > A/'o since A; + <; - 1 < 
Ki(A^). We can upper bound the spectral norm of matrix E, i.e., ||-E'||2; by the Frobenius 
norm of E, i.e, 

II-EII2 = ^ I ^ afe+m-i.jttfe+i-i.i I (325) 

m^l \ i=l / 

2(1 + B.]^ ' 



< ( ) (326) 



,4(1 + 55)2 ^ ^ 

< X' ^ ^2 (327) 



^-2 



< f'-^ (328) 



where ( 13261) follows from (I316p . Hence, we may conclude that 



where the right hand side is a positive number, by the definition of c?2. Next, we evaluate 
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I I |2 
V . 



i=0 
<:-l 

1=0 



'k+i '-'fc+f 



k+i 



AN) 
'k+c; 



< 



'~'k+<; 



do 



(TV) 
k+i 



j=0 



C(l + S5) II (^)| 



< 
< 



do 



AN) 

AN) 
'k+i 



(330) 
(331) 

(332) 

(333) 
(334) 
(335) 



where f l332D follows from fl309|) . and fl333D follows from fl2571) when N > Nq since + ^ - 1 < 
Ki{N). Finally, we evaluate ||b||^. 



^k+i "k+i J 



i=0 



< ^ 

< X 

<x 



{N -Ip 

ABi{K^{N) + B,)'-T^+ '^' ' 
(AT - 1)7 



(336) 

(337) 
(338) 
(339) 



where f l337l) follows from fl275|) when > 
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Following from fl32T|) . using (I335D . (I339|) and fl329D . we have 



< 



l^-^ll^dlbll + llvllf 

^(d2-l)(l-55) 2x(l + fi5 



(340) 



do 



do 



-1 



/x(l+55) 




/ d2 





^ ^^fABl{K,{N) + B,rTl' 



(AT - 1)7 



< 



■(rf2-l)(l-55) 2x(l+55 



d2-l 
do 



do 



do 



-1 




X(l + 55) 



Ci2 



(341) 

(342) 
(343) 



where ( I342p follows when is large enough, due to the fact that Ki{N) satisfies (I207p . i.e, 
there exists an integer N^, such that when N > Nr,, 



(iV-1) 



do 



1 - 



(344) 



and (13421) is true, and (13431) follows from the definition of d2 by (I286p . Hence, when is large 
enough, more specifically, when N > max{No, N2, N3, N^, N^, Nq), we have a contradiction 
with (I305p . Therefore, we conclude that there must be at least x eigenvalues of S'^ within 
distance (13011) away from any of the clustered As, furthermore, from the definition of a cluster 
in (12871) . there must be at least x eigenvalues within distance 



(2x- + 1) 



dodi 



2"0" 



j k + X + B',: 
(AT - l)/3 



(345) 



away from all of the clustered As. We pick x eigenvalues of S'^ which are within distance 
(13451) and arbitrarily pair each clustered A with one of the eigenvalues. These eigenvalues 
will not be paired with any other A because all other clusters of As are at least distance 
2y/d2do ^'''('n^uI^ apart from this cluster. 



(Af-l)/s 

Finally, by letting 

di = (2x + 1) y^^rfo, Bt = x + B', (346) 
we have the desired results when A^ is large enough, i.e., A^ > max(A'o, A^i, A^2, -^^3, A^4, A^s, A^g)- 
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9.9.2 Proof of Lemma [8] 

In the proof of Lemma [HI we will need results from Lemma [6] and Thus, we will first prove 
that under the condition of Lemma [HI the results of Lemma [6] and [7] apply. Since 



lim i^fr =0 

Af^oo 



(347) 



for any < k < 1, when is large enough, 6' < i)^ is small enough, which means that 
the result of Lemma [6] is valid. Now we show that the result of Lemma [7] is also true. Let 



+ Cu- Because of 



lim — - — ^ 



0, lim — — 

Li 



(348) 



we have f[2n71) and (12081) . Because of (1347!) and the fact that <d^, we have (12061) . 



Hence, for any < A; < 



{fr)"^ + Cu , result of Lemma [7] applies because 



k < 



d 



(349) 



and is large enough. 

Now, we will use the result of Lemma [6] and [7] to prove Lemma [8] From the properties 
of the Karhunen-Loeve expansion, we know that 



fc=0 



To 



K{t,t)dt < oo 



(350) 



Thus, for any constant < k < 1, we have 



^)^+c.| 

'J oo 



k=0 



E 



k=0 



Cu\+1 



To 

> I K{t,t)dt 

'0 



k=\[fr]^+cu\+l 



r° d^ 

> / K{t,t)dt- , \. e'^- 

Jo 



{x — 1)k 



(351) 



(352) 



(353) 



where we have used ([73]) in Lemma [6] to obtain (I353p . 
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From the definition of matrix E /v , we liave 



N-l 

E 

k=0 



tr(S 



Tn 



N-l 



N, 



N 



1=0 



Tn 



(354) 



Tlius, 



N-l 



N-l 



{NY 



(355) 



' +Cu +1 



k=0 



k=0 



N-l 



Tq \ ^ / t t 

di 



i=0 



N-l 



- N-l^ \N-1 "'iV-1 " 



(356) 



fe=0 



i=0 



To 



K{t,t)dt 



+ 



-6' 



(357) 



A:=0 



< ^ 1 — h -. ^ — U ^ + 



< 



(N-l)" N-l {x-1)k 
5T^i+"2f +ToJs:(0,0) d^ 



(i^)*+c„J+B7 



fc=0 



(358) 



+ 



{N-iy 

di 



(r + l)(iV- l)/3 



+ 



d 



[x — 1)k 



9' 



T+l 



By 



(359) 



^ i?T^°2f +Toir(0,0) ^ ci^ ^ 



(iV-1)^ 



< 



d' 



70 



/I- 



(360) 
(361) 



{x — 1)^2 

where f l356p follows by Lemma [71 We have used f l353p to obtain (13571) . and condition 2 in 
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Section [3] to obtain (13581) . (13591) follows from the fact that a < 1 and that 

^ i-n -I 

^A;^< / y^dy = -—^n^+^ (362) 

fc=0 ^ 



(13601) follows because (13471) and when is large enough, we have 



(13611) follows because 



c« + 57<^<^ (363) 



lim — — — = (364) 

1j 



and 

lim ^ = ^ hm — ^ = (365) 
and when N large enough, i.e., there exists a N^i^n) > such that when N > N^i^n), we have 

BT^+"2^+ToK{Ofi) BtJ+"2T+ToX(0,0) 

^^Zil: < (^ZDI < 1 f 1 _ 1 ] (SQQ) 



T+l 

„, ,21+1 2did^r- 

2did X i 



(r+l)e'^(7V-l)^ ^ (-+1)(<) - jN-l)^ < 1 _ 1' j (367) 



Therefore, for any < k < 1, (12111) holds for 6' G [d^, when is large enough. 
9.10 Proof of Lemma [4] 

Since the condition of Lemma H] is the same as Lemma [H the results of Lemma [6l [7] and [8] 
hold. By the same argument as Lemma [6l Lemma [5] holds as well. 
We first prove (jSTI) . Since satisfies 

lim ^ = lim -i— = (368) 
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when N is large enough such that 



+ 1 < 



d 



1<N -1 



(369) 



we can provide an upper bound on {6') by sphtting the sum of variables into two parts, 



R 



N (ni\ 



k=0 \ / 



Cu\+1 



For any < k < 1, we start with the first term in (13701) . 



fc=0 



0' 



< 



< 



< 



< 



< 



< 



fc=0 



fc=0 



6' "e'{N-i)(^ 



6' J 2e'{N - 1)^ 





y2 + x 




2k 




y2 + x 




2k 




;2 + X 


2k' 



d^6' ^ + 



di 



2{T + i)e'{N -ly 



E + 

k=0 

d 
J' 



r+1 



+ CU + B7 



{T + i)e'^iN -ly 



d-e'-- 



(370) 



(371) 
(372) 

(373) 

(374) 

(375) 
(376) 



where (I37ip follows from Lemma [71 fl372p follows because the derivative of the function 
|log(l + x) is bounded by | for x > 0, fl373p follows from the definition of the sequence 
A'^' in (fTTil and the observation in (fT2l) . f l374p follows because of (|7ill in Lemma El and the 
observation in (13620 . (I375P follows because of the same reason as (I360p . and (I376P follows 
because of (I365p . and when is large enough, more specifically, there exists an Nq{k) > 
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such that when N > NJk), we have 



did 



T + l 

did X 

T + l + X 



(377) 



Now, we will study the second term of (13701) . Using Jensen's inequality [24], the second 
term of (13701) is bounded by 



N-l 

E 



log 1 



(NY' 



(378) 



I " +c„ +1 



N 



< 



log 



1 + 



N 



N~l 



{NY 



' +Cu +1 



< 



2{x - l)/s:2 



(379) 



(380) 



where in obtaining (I380p . we have used (121 ip in Lemma [8] and the fact that log(l + x) < x. 
We combine the results of (13761) and (13801) and obtain 



d-. (x2 - (1 - log 2)x + (1 -log2)) ^,_i 



2{x - l)/?^ 

Using similar methods, we may also lower bound {6'). We write 



(381) 



^ 1 / /,(^)'' 



+ 



Af-1 



1 



log 1 + 



(NY' 



fc=0 



(382) 
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We start with the first term of (I382p . 



E 



log 1 + 



k=0 

, 1 



{k + Br 



> 



I — n \ / 



> 



fe=0 



E 

fe=0 



di 



log 



di 



> 6 ^ 



> O ^ 



0' J 29'{N - ly 



di 



2{t + i)e'{N -ly 



(383) 



fc=0 

(384) 

E (385) 

T + l 



k=0 



Cl 



did' 



[t + 1)6'^-^ {N - If 



(386) 
(387) 

(388) 



where (13831) follows when applying the result of Lemma [3 fl384p follows because the function 
\ log(l + x) has derivative bounded by |, when x > 0. The first term in (13861) follows because 
of ( 1721) in LemmaO The second term follows because of (I362p . (138 7p follows because of similar 
reasons as (I360p . and (13880 follows because of (I365p . and when is large enough, we have 



T + l 

did X 



did' 



{r+l)e'^^{N^ ^ (r+l)«) - iN-l)0 ^ 1 



xdx Qt-L xdx (-(9^)"^ ° 

A lower bound on the second term of (13820 is zero. Hence, we can conclude that 



rN(0') > ^e'--x 



(389) 



4 



(390) 
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Now we evaluate D^{6') for large enough N and 6' G ["(^l, and prove (!52|) . 



Af-l 



/c=0 \ 



1 1 

+ 



iV-l 



fe=0 



To- E 



1 1 

7T7 + 



r +c„ +1 



(NY 



(391) 



(392) 



where fl392p follows because of the same reason as fl370p . The first term of (13921) can be 
bounded as 



1 1 



k=Q 

■ d ^ 



<r„-' E 

fc=0 

<r„-' E 



+ 



1 



-1 



^A: + "1 (Ar_i)/3 



+ 



fc=0 



6*' \ll ^ rl ^h±ML 
\ + "1 (Ar-l)f 



fe=0 



.9' 



< T,^ (^'' + 



To{N-iy 
di 



rp-l 



k=0 



d 



- To 



2did 



r + 



1+1 



(r + l)To{N - 1)^6^ 



:r + l)To(iV-l)/3 



k=0 

E 

fc=0 



d 



r+l 



5l 



,T + 1 ' rp 

X In 



(393) 
(394) 
(395) 

(396) 

(397) 
(398) 
(399) 



where (13931) is true because of Lemma [71 (13951) follows because the derivative of the function 
(Jy + ^) ^ is bounded by 1, and (I396p follows from the fact that for a, 6 > 0, + ^) ^ < 
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min(a, b). (13971) follows because of fl362p and the fact that when K = (^) + c^j 



(400) 



(13981) follows because of the same reason as (I360p . and finally (I399P follows because (13650 
and (I347p and when is large enough, i.e., for any < k < 1, there exists Nj{k) > such 
that when > Ni{k), we have 



T + l 



2d, 



{t+1)To{N-1)P9'^ ^ (T+l)To(jV-l)/^«) 



dx nll-- 
To 



1 A 
< - - - 1 

- 2\k 



Th < To U < i 

.1 ^ , - .1 -- ^ 1 - 2 V K 



N-l 



To-' E 



-1 



The second term of (13920 can be bounded by using Jensen's inequality. 



< 



( 



N 
< — 
- To 



Tn 



1 



1 

7^ + 



V 



1 






-1 



E 



^1 fc= 



(N)' 



kN 



+ 1 



(TV)' 



-1 



6' 



< ^ min I 



< 



{x — 1)Tqk^ 



(401) 
(402) 



(403) 



(404) 

(405) 
(406) 



where (I404p follows because (13680 and when is large enough, i.e., there exists an Ns{k) > 0, 
such that when N > Ngi^n), we have 



^ g, ) + Cu 



+ 1 



< 



+ Cu 



+ 1 



< 1- K 



(407) 



N - N 

(14051) follows from (12111) in Lemma [8] and (I406P follows because of (I368P and when A^ is large 
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enough, we have 



1 /I 1 



{x-1)kS N {x-1)k3 N ^ 

0' - W) - 

Thus, combining (13991) and (14061) . we have 



(408) 



K-^{x - l)To 



(409) 



Therefore, for any < k < 1, (!5T!) and (152!) are true for 9' G [i^Ly''^u] when N is large 
enough. 



9.11 Proof of Theorem 

Note that (!5T|) imphes that 



'^''^^d J, /'rft(x2-(l-log2)x + (l-log2)) 

- <9a [^) < 



R- 



(410) 



for large enough N and R in the interval of 



(X^ - (1 -log2)x+ (1 -log2)) ^ '^^fqNY 



From the definition of Da{R) in (jlSD, we have 

D^{R) = D^ie^iR)) 

< 2A(^) + + D^{9^{R)) 



< 2A(^) + + 



d-. (l + /t^(x- 1)) 
/«='(a;-l)To 



Km''' 



(411) 



(412) 
(413) 

(414) 



< O (iV"") + O + ^(1 + /^^(x - l))Jx^ - (1 - log2)x + (1 - log2))^-^ ^^i_. 



roK2^+i2^-i(x- 1)= 



(415) 



where f l4T3D follows from ([39]), flM follows because of ([52]), fl4T5D follows from (|48]). (1^. 
(HTOll and the fact that i? in ([55]) implies that is in ( HTTl) . and when is in ( ]4TTil . (i?) 
is in [i^Ly^u]- When R is in (1551) . we have that the third term in (14151) is much larger than 
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the sum of the first and second terms when N is large enough due to the fact that 

lim ]^.U2U = ^ hm ^ = (416) 

i.e., there exists an N<^{i^) > such that when N > Ngi^n), we have 

O (A^-°) + O (ArV2-«) 

IqTk^ (^-l))(a,2-(l-log 2)x+(l-log 2))"-^ Di_^ 
To2^-i{a;-l)^ 

< ^ ^ (417) 

d{l+K^ (x-l))(x2-(l-log 2)a^+(l-log 2))"-^ fxd_ / niV \-^\ 
ro2^-i(2:-l)^ \^ 8 y^LL) J 

(418) 



Therefore, for < k < 1, fl54j) is true for R in the interval of fl55|) when is large enough. 

9.12 Proof of Theorem [6] 

Pick the sequences 'd^j^ and 'djj as 

= ( J^iogArp(Ar) , d^=\ \ogNP{N)\ (419) 

1 xdi ^ ' ] ^ \ d^(x2-(l-log2)x+(l-log2)) ^ ' ] ^ ' 

\ 8 / \ 2(x-1)k2 / 

Then, because P(A^) satisfies (l43l) and ( 156|) . "(9^^ satisfies ( l53l) and "(9^^^ satisfies ( l50|) . Accord- 
ing to (HI]), we have the achievable rate in the interval of ( l55l) . and thus, when A^ is large 
enough. Theorem [5] applies. Hence, an upper bound on the minimum achievable expected 
distortion, or equivalently, the achievable rate in the separation-based scheme is 

= (O (420) 

< ^(1 + - 1)) (x^ - (1 - log2)x + (1 - log2))-^ ( 1 \ 

Tqk^^+^2^-\x - lYv^-^ \\og{NP{N))) ^ ' 

Therefore, when P{N) satisfies (H3ll and ( l56l) . for any < k < 1, ( l58ll holds when A^ is large 
enough. 
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